Search for: All records

We present WonderWorld, a novel framework for interactive 3D scene generation that enables users to interactively specify scene contents and layout and see the created scenes in low latency. The major challenge lies in achieving fast generation of 3D scenes. Existing scene generation approaches fall short of speed as they often require (1) progressively generating many views and depth maps, and (2) time-consuming optimization of the scene representations. Our approach does not need multiple views, and it leverages a geometry-based initialization that significantly reduces optimization time. Another challenge is generating coherent geometry that allows all scenes to be connected. We introduce the guided depth diffusion that allows partial conditioning of depth estimation. WonderWorld generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU, enabling real-time user interaction and exploration. Our interactive demo, full code, data, and software can be found at https://kovenyu.com/WonderWorld/ 

We introduce AutoVER, an Autoregressive model for Visual Entity Recognition. Our model extends an autoregressive Multimodal Large Language Model by employing retrieval augmented constrained generation. It mitigates low performance on out-of-domain entities while excelling in queries that require visual reasoning. Our method learns to distinguish similar entities within a vast label space by contrastively training on hard negative pairs in parallel with a sequence-to-sequence objective without an external retriever. During inference, a list of retrieved candidate answers explicitly guides language generation by removing invalid decoding paths. The proposed method achieves significant improvements across different dataset splits in the recently proposed Oven-Wikibenchmark with accuracy on the Entity seen split rising from 32.7% to 61.5%. It demonstrates superior performance on the unseen and query splits by a substantial double-digit margin, while also preserving the ability to effectively transfer to other generic visual question answering benchmarks without further training. 

Realistic object interactions are crucial for creating immersive virtual experiences, yet synthesizing realistic 3D object dynamics in response to novel interactions remains a significant challenge. Unlike unconditional or text-conditioned dynamics generation, action-conditioned dynamics requires perceiving the physical material properties of objects and grounding the 3D motion prediction on these properties, such as object stiffness. However, estimating physical material properties is an open problem due to the lack of material ground-truth data, as measuring these properties for real objects is highly difficult. We present PhysDreamer, a physics-based approach that endows static 3D objects with interactive dynamics by leveraging the object dynamics priors learned by video generation models. By distilling these priors, PhysDreamer enables the synthesis of realistic object responses to novel interactions, such as external forces or agent manipulations. We demonstrate our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study. PhysDreamer takes a step towards more engaging and realistic virtual experiences by enabling static 3D objects to dynamically respond to interactive stimuli in a physically plausible manner. See our project page at this https URL. 

Volcanic eruptions deposit Fe-bearing volcanic ash in the ocean, thereby increasing biological productivity. The increased organic matter in areas of high biological productivity uses up oxygen as this organic matter decays and sinks through the water column. Past living beings, like foraminifera, ate organic matter that was carbon-rich and sometimes had metals absorbed into their carbon, creating coatings inside and outside their shells. These coatings can tell us about how biological productivity was affected before, during, and after the volcanic eruption. The studied cores are from the northwest Pacific Ocean and are close to geologically young volcanoes that are not well understood. The two cores that we focused on were VM28-309 and VM36-15 both taken by the Vema research ship. We studied the relationship between ash deposition and biological productivity by looking at all the ash layers in both cores. We found that in most of the ash layers, there were black or dark-colored foraminifera with coatings inside and outside the shells that were often carbon-rich and sometimes metal-rich. We attribute this coating to the increase of organic matter in surface waters when there was deposition of large amounts of volcanic ash. We also found high concentrations of Barium metal in VM28-309. Barium (Ba) is a biological marker because most or all Ba originates from the organic matter contained in sediments. We found that ash layers containing the finest materials (<38 micrometers in size) had the highest Ba content. For accurate results, we must sample above and below ash layers and select more sediment cores in the area. Also, Barium corrections must be done using data on biogenic silica contents. Loss on ignition (LOI) data will give us an estimate of the total organic carbon in each sample- allowing a second direct assessment of the increase in biological productivity produced by the deposition of volcanic ash. 

Known as a bio-limiting metal, high abundances of iron in sea water can amplify biological productivity. The growth of diatoms and other photosynthetic organisms increases, providing more food for grazing organisms like foraminifera. The net result is more organic matter in surface waters and ultimately in surface sediments. Existing satellite data show increases in ocean chlorophyll in areas affected by volcanic eruptions. We infer from this that iron derived from volcanic ash does increase biological productivity. However, the relative increase in productivity is unknown. We examined 3 sediment cores from the Equatorial Western Pacific to analyze the relationship between volcanic ash and biological productivity: RC14-44, RC14-66, and RC14-67. All contain black or dark-colored foraminifera within ash layers and white-shelled foraminifera outside ash layers. We attribute the dark material outside and inside the foraminifera to organic carbon and metals. In our cores, some foraminifera are covered in iron sulfide (FeS), which could be pyrite, and contain large amounts of carbon as well as high abundances of aluminum and silicon. We examined barium concentrations to gain further knowledge of biological productivity at specific core depths as barium is a marker for primary productivity. We found that barium levels within ash layers increased at least ten-fold. Within ash layers, we also noticed that the ashes with higher amounts of fine silt and clay sized material have the greatest increase in barium content, perhaps related to explosion size. This pattern of increases in Ba, metals and organic carbon within ash layers compared to surrounding sediments shows that volcanic ash deposition increases marine productivity. For future research, measuring markers for biological productivity like biogenic silica content and loss on ignition (LOI) within and outside ash layers would further clarify the relationship between volcanic ash deposition and biological productivity.